Touch screen control

ABSTRACT

This document relates to touch screen controls. For instance, the touch screen controls can allow a user to control a computing device by engaging a touch screen associated with the computing device. One implementation can receive at least one tactile contact from a region of a touch screen. This implementation can present a first command functionality on the touch screen proximate the region for a predefined time. It can await user engagement of the first command functionality. Lacking user engagement within the predefined time, the implementation can remove the first command functionality and offer a second command functionality.

BACKGROUND

Computing devices continue to evolve into different configurations, suchas notebook computers, personal digital assistants (PDAs), cellphones/smart phones, etc. Traditionally, computing devices employedseparate input devices, such as mice and/or keyboards and outputdevices, such as a display screen. Various configurations have beendeveloped and refined to allow the user to easily control the computingdevice in these traditional scenarios. For instance, some of theseconfigurations employ a toolbar of commands at a set location on thedisplay and the user can control the computing device by selectingcommands with an input device such as a mouse.

Relatively recently touch screens have become more readily available.Touch screens blur the traditional lines by offering the capability tofunction as both an input device and an output device. Adapting thetraditional configurations to touch screen scenarios tends not toproduce high user satisfaction and/or underutilizes the capabilitiesoffered by the touch screen.

SUMMARY

This document relates to touch screen controls. For instance, the touchscreen controls can allow a user to control a computing device byengaging a touch screen associated with the computing device. Oneimplementation can receive at least one tactile contact from a region ofa touch screen. This implementation can present a first commandfunctionality on the touch screen proximate the region for a predefinedtime. It can await user engagement of the first command functionality.Lacking user engagement within the predefined time, the implementationcan remove the first command functionality and offer a second commandfunctionality.

Another implementation can receive a first tactile contact from a useron a touch screen. The implementation can present a toolbar of commandswhile the receiving continues. The implementation can receive a secondtactile contact from the user over an individual command of the toolbar.The implementation can provide a corresponding command functionality.This implementation can also allow the user to control the correspondingcommand functionality by sliding the first tactile contact and thesecond tactile contact on the touch screen.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key or essentialfeatures of the claimed subject matter, nor is it intended to be used asan aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of similar reference numbers in different instances in thedescription and the figures may indicate similar or identical items.

FIGS. 1-23 are examples of touch screen control usage scenarios inaccordance with some implementations of the present concepts.

FIG. 24 is an example of a system that can implement touch screencontrol concepts.

FIGS. 25-26 are flow diagrams of touch screen control methods inaccordance with some implementations.

DETAILED DESCRIPTION

Overview

More and more, computing systems are sold with touch screens. Touchscreens accept user input based on physical or tactile contact. A usercan achieve this tactile contact by engaging contact detectors includedin the touch screen with his/her finger(s), a stylus(es), or othermechanism. The present concepts allow a user to call up a commandarchitecture by engaging the touch screen. Further, the commandarchitecture can be presented proximate to the user engagement, ratherthan at a fixed location, including—but not restricted to—beingsuperimposed in a semi-transparent form directly over the area ofengagement. On both mobile form-factors and large-screen formats(tabletops, whiteboards, etc.) it can be desirable to activate commandsin-place or proximate to the locus of interaction. On small mobiledevices this proximity can have the virtue of avoiding permanentlyvisible icons, palettes, or other widgets. On large format displays thisproximity can have the virtue of avoiding round trips to a distant userinterface control, and, by appearing at a location dictated by theposition of the user's hand, thereby ensuring that controls are withinreach of the user (in contrast to appearing in a menu bar at the top ofa large wall-mounted display where they may well be out of reach of manyusers).

The present concepts can also attempt to distinguish user intent whenthe user engages the touch screen. Toward that end, a first commandfunction can be offered to the user responsive to the user engagement.For example, the user touches the touch screen and a toolbar ispresented proximate the touch. A determination can be made whether theintended command function was offered to the user. For instance, thedetermination can be based upon user response to the offered commandfunction. Continuing with the above example, if the user does not selecta command from the presented toolbar, then it is likely that the intentof the user remains unsatisfied. An alternative command function can beoffered to the user if it is determined that an incorrect commandfunction was offered. For example, the toolbar may be removed andcontent near the user touch may be highlighted.

Example Scenarios

FIGS. 1-23 illustrate exemplary manifestations of the touch screencontrol concepts described in this document.

FIGS. 1-5 collectively illustrate a first touch screen controlimplementation. FIGS. 1-5 include a computing device or “device” 102(1)that has a touch screen 104(1). User interaction with the touch screenis represented by a user's hand 106(1).

Initially, in FIG. 1 the user's hand 106(1) is proximate but notcontacting touch screen 104(1). Content in the form of “Giraffes are . .. ” and “Zebras are . . . ” is displayed on the touch screen.

In FIG. 2, the user's hand 106(1) engages the touch screen 104(1). Inthis case, the user's fingertip engages the touch screen proximate tothe “Giraffes are . . . ” content as indicated at 202.

In FIG. 3, the device 102(1) can attempt to determine the user's intentassociated with the fingertip engagement of FIG. 2. In this example, thedevice 102(1) offers a first set of commands 302 to the user. In thiscase, for purposes of explanation, the first set of commands 302includes a ‘back arrow’ and a ‘forward arrow’ or ‘next arrow’. In thisimplementation, the first set of commands are offered proximate to thefingertip engagement. In this case, “proximate to” means that thelocation of the first set of commands is based at least in part upon thelocation of the finger engagement rather than being at a predeterminedfixed location, such as the top or bottom of the touch screen. In someof these cases, “proximate to” can mean at or adjacent to the locationof the finger engagement. In other cases, the location of the first setof commands may be farther from the location of finger engagement, butstill based at least in part upon the finger engagement location.

FIGS. 4 and 5 offer two alternative scenarios that can occur responsiveto the scenario of FIG. 3. In FIG. 4, the user, via user's hand 106(1),engages a command of the first set of commands 302. In this example, theuser engages the ‘back arrow’. Thus, in this scenario the deviceaccurately predicted the user's intent and offered appropriate controloptions to the user.

FIG. 5 illustrates an alternative scenario where the user does notengage any of the offered set of commands 302 within a predefined timeperiod. In this case, the device 102(1) can interpret the user's lack ofengagement of the offered commands as an indication that the user'sintent has not been satisfied. In such a scenario, the device can removethe first set of commands and offer another command functionality. Insome implementations, the first set of commands can be removed bygradually fading them out over a period of time, such as one second. Inthe example of FIG. 5, the device highlights displayed content that isproximate to the contact (e.g., “Giraffes are . . . ”) (now bold,underlined text) for the user and can automatically conduct a web-searchon the highlighted text for the user. (The results of the web-search arenot illustrated here). The above discussion illustrates how the devicecan attempt to offer desired commands to the user and can update theoffered commands based upon the user's response.

FIGS. 6-8 collectively illustrate a second touch screen controlimplementation.

Initially, in FIG. 6 a user's hand 106(2) engages touch screen 104(2)with two contact points 602(1) and 602(2) at substantially the same time(such as +/−0.5 seconds). Device 102(2) can calculate an initialdistance D between the two contact points 602(1) and 602(2). The devicecan attempt to determine user intent from the two contact points. Forinstance, if the two contact points are touching one another, ordistance D is less than a predefined distance or defined value, thedevice can interpret the user intent as being different than an instancewhere the contact points are at a distance that is greater than distanceD. For example, the predefined distance can be 0.5 inches. In such acase, two substantially simultaneous contacts within 0.5 inches can betreated differently than those equaling or exceeding 0.5 inches.

FIG. 7 illustrates a subsequent view where the device displays a firstcommand set in the form of toolbar 702 responsive to the two usercontact points discussed above relative to FIG. 6. In this instance,assume that the distance D is less than the predetermined value.

FIG. 8 shows a subsequent view where the user maintains contact with thetouch screen 104(2) and spreads the contact points away from oneanother. In this example the user's thumb slides downward as indicatedby arrow 802. Alternatively, the predefined distance could be altered bymoving the index finger and/or both the index finger and the thumb couldalso be moved, to essentially the same effect. In response to such slideaction, the toolbar 702 can be extended, offering additional commands804(1), and/or, offering additional toolbars 804(2) as indicated in thisexample.

Thus, a mechanism is provided that not only enables the presentation ofadditional commands, but in which those commands can be organized byfunctional groups as indicated at 804(1) and 804(2). While an initialtoolbar and two additional toolbar functionalities are illustrated, anynumber of toolbars or toolbar functionalities can be generated for theuser responsive to the slide action. Further, in this example thetoolbar 702 is expanded via two organized functional groups 804(1) and804(2). In other implementations, expanding toolbar 702 may simplyincrease the number of displayed icons that are listed on the toolbarwithout the toolbar being broken into discrete sub-units. In a furtherconfiguration the expanding toolbar could maintain highly used commandsin the initial portion of the toolbar and add additional commands as thetoolbar is expanded.

In this case the toolbar 702 is expanded in what can be termed an‘accordion fashion’ that follows the movement and position of the usercontacts (e.g., the toolbar is adjusted in a manner that corresponds tothe user action). Other implementations can expand the toolbar in otherways, such as in a radial fashion, or only horizontally (e.g.,left-to-right) rather than following both horizontal and verticalsliding. Recall that in this case, the device 102(2) determined based atleast in part on distance D that the user intent related to calling up atoolbar. In an instance where the distance D exceeded the predeterminedvalue, then the device may have offered other command options to theuser. For example, the device may have determined that the user wantedto select and zoom content between the two contact points (e.g., “zoomfunction”).

Alternatively or additionally to deriving user intent from initialdistance D, user intent can be inferred from the slide action. Forinstance, if the user maintains the contact 602(1) stationary whilesliding contact 602(2) (e.g., the thumb) away, the technique caninterpret that the user intent is to expand the toolbar. In contrast, ifthe user slides both contacts 602(1) and 602(2) away from one another,the technique can interpret that the user intent is to zoom on contentbetween the contacts.

In summary, the present implementations can offer enhanced commandfunctionality to a user of the device 102 and attempt to offerindividual command functionalities to match user intent. Further, someof these implementations can utilize subsequent user actions to furtherrefine the presented command functionality. Also, while a two fingerpinch example is illustrated, other implementations can utilize otherpinch gestures, such as a three finger pinch gesture.

FIGS. 9-16 collectively illustrate a third touch screen controlimplementation that is similar to the second touch screen implementationdescribed above.

Initially, in FIG. 9 a user's hand 106(3) engages touch screen 104(3) ofdevice 102(3) with a single contact point 902. In this case, the singlecontact point is made by the tip of the user's index finger. Someimplementations can measure the size of the single contact point andutilize the size as a parameter for determining user intent. Forinstance, a large contact point, such as where the user's palminadvertently contacts the touch screen can be interpreted differentlythan a fingertip size contact point. Alternatively or additionally, theorientation and/or shape of the contact point can be used as parametersto determine user intent.

FIG. 10 shows a subsequent view of device 102(3) where a toolbar 1002 ispresented proximate to the location of the single contact point 902. Afinger shadow 1004 is also presented for the user proximate to thesingle contact point. The finger shadow can be presented based upon adetermination of the user's intent with the single contact point. Thepurpose of the finger shadow is explained below relative to FIGS. 11-12.Briefly, a finger shadow can offer the user the option of expanding apresented toolbar without the user having to generate two simultaneouscontacts.

Note also that some implementations can utilize a context of the contactpoint 902 as a factor or parameter for determining user intent. Forinstance, if the contact point is over text or an icon, a first userintent may be determined. In contrast, if the contact point is in a‘blank’ area of the touch screen then a different user intent may bedetermined. Other user actions can also be utilized to determine userintent. For instance, if toolbar 1002 is presented and the user thentaps another area of the touchscreen not associated with the toolbarthen the user may not want the toolbar and the toolbar can be removed.Some implementations may further attempt to interpret this subsequentuser input. For instance, a quick (e.g., less than one second) tap(e.g., in a single location) may be interpreted in a different mannerthan a contact across the screen (drag or stroke). For example, a quicktap may indicate that the user wants the toolbar to be removed, whereasa drag or stroke may indicate intent to utilize a command from thetoolbar.

FIG. 11 shows a subsequent view of device 102(3) where the user hascontacted the touch screen 104(3) with his/her thumb over the fingershadow 1004. Note that some implementations may position a first fingershadow under the first contact point 902 and may position finger shadow1004 to indicate or suggest where the user may make another contact. Insuch a case, the first finger shadow can be associated with toolbar 1002and finger shadow 1004 can be utilized to instantiate additionalcommands.

FIG. 12 shows a further view where the user is sliding his/her thumb andindex finger apart to expand the toolbar in an accordion fashion similarto that described above relative to FIGS. 6-8. However, in this case,the user does not have to make the simultaneous two fingered contactgesture described above. In this implementation, the option of toolbarexpansion is provided to the user should the user so decide to utilizethis command. If the command the user desires is in the initial toolbar1002 presented in FIG. 10, then the user can ignore the finger shadow1004. In such a case, the finger shadow obscures little or no screenspace. Further, some implementations can automatically fade out thefinger shadow if the user does not contact it within a predefinedduration of time.

The finger shadow 1004 can be utilized in alternative or additionalways. For instance, in one configuration, a user contact with the fingershadow can function to activate a specific command, such as a lassoselection tool, or a last (e.g., most recently) used command. Thus,responsive to the initial contact the finger shadow is generated. If theuser contacts the finger shadow and slides his/her fingers apart thenthe user can expand the toolbar in the accordion fashion describedabove. The finger shadow can enable this feature without simultaneouscontacts. In some implementations the finger shadow can also enable analternative functionality. For instance where the user engages, but doesnot slide the finger shadow then another command can be activated. Inone implementation, the activated command is the lasso tool. Byactivating the lasso tool via the finger shadow, the user canimmediately start to select content by engaging (e.g., touching andholding) the touch screen with his/her other hand or with a pointingtool in the other hand. In some implementations this touch-and-hold canimmediately activate the secondary mode (e.g. lasso). In this case,there no time-out, as there is for traditional touch-and-hold contextmenus, for example. The finger shadow may also function as a clip boardwhere content can be stored by dragging the content to the fingershadow.

In some implementations the initial contact can be utilized for the mainmodal controls of an application (such as lasso mode, pen stroke mode,highlighter stroke mode, paintbrush mode, eraser mode, etc.) then thefinger shadow would offer whatever mode was most recently used. Thisconfiguration can provide a shortcut where the user can quickly movebetween drawing normal ink strokes, or, via the finger shadow, makestrokes in whatever mode was most frequently used. Allowing rapidinterleaving of normal pen strokes and any of these commonly used modescan allow the system to efficiently support a variety of common taskworkflows.

In these embodiments, the most recently used mode can be shown asattached to (or overlaid with) the finger shadow itself, and the toolbaritself can be expanded from the finger shadow. The icons on the toolbardo not change order depending on which mode is currently on the fingershadow itself. This can make the interaction with the toolbar morepredictable and habitual for the user.

FIG. 13 shows the user engaging an offered command 1302 with a fingertip of his/her right hand as indicated at 1304. In this case, command1302 is a drawing tool to create an circle shape having a dotted line.In some implementations, the toolbar can remain in its present formwhile the user utilizes command 1302. In other implementations, thetoolbar can be see-though or transparent during the drawing operation.Stated another way, in some implementations, the toolbar is visible atthe onset of the task, disappears/fades out when the user selects theoption, and optionally may fade back in when the user finishes draggingout the indicated shape. FIGS. 14-16 illustrate another configurationwhere the toolbar disappears during the drawing operation and reappearsat the conclusion of the drawing operation.

FIG. 14 shows a view immediately subsequent to the user selectingcommand 1302. The toolbar has been removed responsive to the userselecting the circle tool to provide an uncluttered work area on thetouch screen 104(3).

FIG. 15 shows the user sliding his/her right finger 1304 to create thedesired circle 1502 on the touch screen 104(3). A similar functionalitycould be achieved by sliding one or more fingers of the left hand and/orfingers of both the right and left hand. For instance, the left hand canbe used to control the center of the circle and the right hand can beused to control the diameter of the circle. (Of course, moving thecenter can also affect the diameter and/or location of the circle).

FIG. 16 shows a subsequent view after the user has completed the circle1502 and disengaged his/her right finger 1304 from the touch screen104(3). In this case, toolbar 1002 is again visualized on the touchscreen responsive to the disengaging so that the user can select anothercommand.

Some implementations can allow the user to redefine what the “default”or top-most toolbar commands are for the pinch-to-expand-commandsgesture. For example, the user can pinch-to-expand all the commands,which are divided into several functional sets. If the user keeps thethumb in contact with the screen, lifts the index finger only, and thenbrings the index finger down on a different tool palette, then thatpalette can become the top-most one that is presented first inconjunction with the finger shadow and/or the quick-access toolbar. Inthis way, the user may optimize which tool palettes are closest at handto make particular tasks.

FIGS. 17-23 collectively illustrate a fourth touch screen controlimplementation.

FIG. 17 shows a device 102(4) that has a touch screen 104(4). Further,FIG. 17 shows a user's left and right hands 1702 and 1704. Further, theuser is grasping a pointing device, such as a stylus 1706 in his/herright hand.

FIG. 18 shows the user making a single contact on the touch screen104(4) with his/her left index finger as indicated at 1802.

FIG. 19 shows a toolbar 1902 generated responsive to the user contactdescribed above relative to FIG. 18. Note that in this implementationthe toolbar is generated proximate to the location of the user contact.

FIG. 20 shows the user selecting an individual command from the toolbar1902 utilizing the stylus 1706. In this case, the individual command isa straight line drawing tool 2002. Of course, a multitude of othercommand functionalities can be offered. In this implementation,selection of the straight line drawing tool activates the tool so thatdrawing can occur without requiring further user actions to begindrawing. This aspect is illustrated in FIG. 21.

FIG. 21 shows the user having moved the stylus 1706 from its initialposition on the straight line drawing tool 2002 to a second positionthat is downward and to the right on the drawing page. Viewed from oneperspective, the user has created a line 2102 that extends between twoendpoints 2104(1) and 2104(2). The second endpoint 2104(2) can bethought of as being defined by and co-occurring with the location of thestylus 1706. Thus, the line 2102 can be partially defined by thelocation of the second endpoint 2104(2), which itself is defined by thelocation of the stylus. This implementation also allows the user todefine the first endpoint 2104(1) of line 2102 utilizing the contact1802 which in this case is made by the user's left index finger. Recallthat the toolbar 1902 that allowed the user to create line 2102 was alsogenerated by contact 1802. While not illustrated, some implementationsmay employ a visual cue to aid the user in understanding therelationship between contact 1802 and the first endpoint 2104(1). Forinstance, a colored dot can be presented on the touch screen proximateto the user's left index finger and another dot of the same color can bepresented proximate to the first end point 2104(1). If the user moveshis/her fingertip (described below relative to FIG. 22) the relativeorientation and distance of the two colored dots can be maintained. Insome implementations, the two colored dots (or other graphical elements)may be associated with a colored line or cone that extends therebetween.The function here is to help the user understand the correlation betweenthe finger contact and the first endpoint 2104(1). This function canalso be achieved with other mechanisms.

FIG. 22 shows a subsequent view where the user has maintained contact1802, but ‘slid’ the contact on the screen to a new location 2202.Correspondingly, the first endpoint 2104(1) of line 2102 has beenadjusted to maintain a consistent relative orientation and/or distanceto the contact 1802. In some implementations, the first endpoint couldbe created and maintained directly under the contact 1802. However, inthis implementation, the first endpoint 2104(1) is offset from thecontact 1802 by some value and the value of that offset is maintained asthe user controls the first endpoint 2104(1) by moving the contact 1802.For instance, assume for purposes of explanation, that as shown in FIG.21, the first endpoint is 10 units in the positive x-direction and 5units in the positive y-direction from contact 1802. When the user movescontact 1802 to a new location, the first endpoint is automaticallymoved to maintain the same 10 unit positive x and 5 unit positive yrelative orientation.

FIG. 23 shows another subsequent view to further aid the reader inunderstanding how the user can control properties of line 2102 via firstand second endpoints 2104(1) and 2104(2). In this case, the user hasmoved the contact 1802 produced by his/her left index finger generallyin the positive x direction. Accordingly, the first endpoint 2104(1) hasmoved in the positive x direction to maintain the constant orientationand/or distance relative to the contact 1802. The user has also movedthe stylus 1706 in the positive (e.g., up) y direction. Accordingly, thesecond endpoint 2104(2) has moved with and continues to underlie thestylus 1706.

Note that the offset between the location of the contact 1802 and thefirst endpoint 2104(1) allows the user to control the endpoint withoutobscuring his/her view behind the finger that is generating the contact.Some implementations can determine where to position the endpointrelative the size of the contact. For instance, for a relatively smallcontact, such as can be generated by a stylus, the endpoint can bepositioned directly under the stylus. For a relatively large contact,such as can be generated by a fingertip, the endpoint can be offset.Thus, such a configuration reduces or avoids the user blocking largeamounts of a viewing area of interest with his/her finger. The viewingarea of interest can be thought of including portions of the toolbar,the endpoint itself and/or content displayed on the touch screen. Incontrast, the stylus is relatively thin and tends to block less of theviewing area of interest. Further, the stylus is a relatively precisepointing instrument that allows the user to precisely define anendpoint. A finger on the other hand is relatively large and impreciseand providing an offset can actually allow the user to control theassociated endpoint more precisely than if the endpoint was positioneddirectly under the user's finger. The above examples utilize one fingercontact and one stylus contact. These implementations can also handlesubstituting another finger contact for the stylus or another styluscontact for the finger contact, among other variations.

The above described offset between the finger contact 1802 and the firstline end 2104(1) can result in a situation where the finger contact canreach the edge of the screen before the line does. Some implementationscan further refine these instances where the contact 1802 or the firstline end 2104(1) approaches the screen edges. For example, oneimplementation can define the interaction such that if the user dragsthe finger to the edge of the screen, the mode remains active, and theuser can then lift the finger and return to a “handle” that is left atthe end of the line to continue dragging it to the screen edge. Such astrategy can also allow the user to interleave additional touch gesturessuch as panning/scrolling, or pinch-to-zoom, to adjust the view and thenagain touch the line to continue controlling it.

Please note that in the illustrated example of FIGS. 17-23, toolbar 1902remains visible and unchanged during the drawing process of FIGS. 21-23.In other implementations, the presentation of the toolbar may be changedin some way during the drawing process. For instance, some degree oftransparency may be applied to the toolbar, or the toolbar may beeliminated completely during the drawings process. The toolbar could berestored upon culmination of the drawing process, such as when the userlifts contact 1802 and/or stylus 1706.

Also note that in another implementation, toolbar 1902 could be expandedand controlled by two or more contacts, such as is illustrated anddescribed above relative to FIGS. 11-12 to create an expanded toolbar.For instance, the multiple contacts could be from multiple fingers ofthe user's left hand. The multiple contacts of the user's left handcould be used to control the first line end 2104(1) of the line whilethe stylus (or the user's right hand) is utilized to control the secondline end 2104(2). The skilled artisan should recognize still othervariations that are consistent with the described concepts.

Example Operating Environment

FIG. 24 shows a system 2400 upon which touch screen control techniquescan be implemented. For purposes of explanation, four devices 102(1),102(2), 102(5) and 102(6) are illustrated. In some cases parentheticalsare utilized after a reference number to distinguish like elements. Useof the reference number without the associated parenthetical is genericto the element. Devices 102(1) and 102(2) were previously introduced,while devices 102(5) and 102(6) are newly introduced. For sake ofbrevity, not all of the previously introduced devices are representedhere. Instead, examples of several types of devices are illustratedrelative to system 2400. Further, for sake of brevity only two of theillustrated devices are described in detail. This description can beapplied to other devices. This description is also intended to provideexamples of device configurations that can be employed to support touchscreen control concepts. The skilled artisan will recognize other deviceconfigurations that can support the described touch screen controlconcepts.

In this case, device 102(1) is manifest as a smart phone type device.Device 102(2) is manifest as a tablet type device. Device 102(5) ismanifest as a laptop or notebook computer. Device 102(6) is manifest asa virtual white board where the user can engage the board to controlcontent displayed on the board. The term “device” as used herein canmean any type of device that has some amount of processing capability.While specific examples of such devices are illustrated for purposes ofexplanation, other examples of such devices can include traditionalcomputing devices, such as personal computers, cell phones, smartphones, personal digital assistants, or any of a myriad of ever-evolvingor yet to be developed types of devices. Further, a system can bemanifest on a single device or distributed over multiple devices.

Individual devices 102 can exchange data over a network 2402.Alternatively or additionally, data may be exchanged directly betweentwo devices as indicated, such as via a USB protocol or can be exchangedvia a storage media or storage 2404. Individual devices can function ina stand-alone or cooperative manner to achieve touch screen control. Forinstance, part of a functionality offered on a device may be performedon the device and part may be performed on another device and/or in thecloud 2406. As used herein, the cloud 2406 refers to computing resourcesand/or computing functionalities that can be accessed over network 2402.

A multitude of different configurations can be created to configure adevice 102 to accomplish touch screen control concepts. For purposes ofexplanation, examples of two possible device configurations aredescribed below relative to devices 102(1) and 102(2).

Device 102(1) includes an application(s) 2408 running on an operatingsystem (OS) 2410. The operating system can interact with hardware 2412.Examples of hardware can include storage media or storage 2414,processor(s) 2416, and a touch screen 104(1). Further, a geometrytracking component 2418(1) and a tactile command relation component2420(1) can function in cooperation with application(s) 2408 and/oroperating system (OS) 2410 and touch screen 104(1).

Processor 2416 can execute data in the form of computer-readableinstructions to provide a functionality, such as a touch screen controlfunctionality. Data, such as computer-readable instructions can bestored on storage 2414 and/or storage 2404. The storage 2414 can includeany one or more of volatile or non-volatile memory, hard drives, and/oroptical storage devices (e.g., CDs, DVDs etc.), among others.

The devices 102 can also be configured to receive and/or generate datain the form of computer-readable instructions from storage 2404 that forsake of explanation can be thought of as external storage media.Examples of external storage media can include optical storage devices(e.g., CDs, DVDs etc.), hard drives, and flash storage devices (e.g.,memory sticks or memory cards), among others. The devices may alsoreceive data in the form of computer-readable instructions over network2402 that is then stored on the device for execution by its processor.

Device 102(2) has a configuration that can be representative of a systemon a chip (SOC) type design. In such a case, functionality provided bythe device can be integrated on a single SOC or multiple coupled SOCs.In this case, device 102(2) includes shared resources 2422 and dedicatedresources 2424. An interface(s) 2426 facilitates communication betweenthe shared resources and the dedicated resources. As the name implies,dedicated resources 2424 can be thought of as including individualportions that are dedicated to achieving specific functionalities. Forinstance, in this example, the dedicated resources include geometrytracking component 2418(2) and tactile command relation component2420(2). In one case, circuitry on the SOC can be dedicated to thegeometry tracking component 2418(2) while other circuitry can bededicated to tactile command relation component 2420(2).

Shared resources 2422 can be storage, processing units, etc. that can beused by multiple functionalities. In this example, the shared resourcesinclude touch screen 104(2). While in this case, geometry trackingcomponent 2418(2) and tactile command relation component 2420(2) areimplemented as dedicated resources 2424, in other configurations, eitheror both of these components can be implemented on the shared resources2422 and/or on both the dedicated resources 2424 the shared resources2422.

Geometry tracking component 2418 can be configured to track tactilecontacts at locations on touch screen 104. In some configurations, thetouch screen can include contact detectors which are coupled to thegeometry tracking component 2418. Geometry tracking component 2418 cantrack the contact detectors and their relative states (i.e. activated orinactivated) on touch screen 104. The geometry tracking component canalternatively, or additionally, track various content displayed on thetouch screen, such as graphical windows, icons, etc.

Tactile command relation component 2420 can be configured to correlatetactile contacts detected by the geometry tracking component 2418 tocorresponding commands. For instance, the geometry tracking unit candetect two simultaneous contacts on the touch screen that are proximateto one another. The tactile command relation component can attempt todetermine user intent associated with the two simultaneous contacts andpresent commands that correlate to the user intent. For instance, thetactile command relation component may compare a distance between thetwo simultaneous contacts to a predefined value.

If the distance exceeds the predefined value, the tactile commandrelation component may cause the content between the two contacts to beenlarged or zoomed. Alternatively, some implementations may zoom contentassociated with the two contacts where some of the content is notbetween the two contacts. For instance, the user may make the twocontacts on content within a window. The zooming feature may zoom all ofthe content of the window or just the content between the fingers.Alternatively, if the distance is less than the predefined value thetactile command relation component may present a toolbar proximate thecontacts and allow the user to expand the toolbar by sliding thecontacts apart.

Also, once some implementations decide the user desires to zoom content(e.g., go into zoom mode) the determination based upon the predefinedvalue may be temporarily suspended to provide the user a full spectrumof zoom features. For instance, once the device is operating in the zoommode, the user may pinch and expand regardless of the distance betweenhis/her fingers. The determination can be re-instated when the userswitches to another mode.

In another configuration, geometry tracking component 2418 can detect afirst individual tactile contact on the touch screen 104. The tactilecommand relation component 2420 can be configured to cause at least onecommand category to be displayed on the touch screen proximate to thefirst individual tactile contact. The geometry tracking component candetect a second individual tactile contact proximate to an individualcommand category. The tactile command relation component can beconfigured to cause a plurality of commands belonging to the individualcommand category to be displayed proximate to the second individualtactile contact.

Generally, any of the functions described herein can be implementedusing software, firmware, hardware (e.g., fixed-logic circuitry), manualprocessing, or a combination of these implementations. The term“component” as used herein generally represent software, firmware,hardware, whole devices or networks, or a combination thereof. In thecase of a software implementation, for instance, these may representprogram code that performs specified tasks when executed on a processor(e.g., CPU or CPUs). The program code can be stored in one or morecomputer-readable memory devices, such as computer-readable storagemedia. The features and techniques of the component areplatform-independent, meaning that they may be implemented on a varietyof commercial computing platforms having a variety of processingconfigurations.

Example Techniques

FIG. 25 illustrates a flowchart of a touch screen control technique ormethod 2500.

Block 2502 can receive at least one tactile contact from a region of atouch screen.

Block 2504 can present a first command functionality on the touch screenproximate the region for a predefined time.

Block 2506 can await user engagement of the first command functionality.

Block 2508, when lacking the user engagement within a predefined timeperiod, can remove the first command functionality and offer a secondcommand functionality.

FIG. 26 illustrates a flowchart of another touch screen controltechnique or method 2600.

Block 2602 can receive a first tactile contact from a user on a touchscreen. In some cases, the first tactile contact can be a single tactilecontact, such as from a single fingertip. In other cases, the firsttactile contact can be multiple substantially simultaneous contacts,such as two, three, or more fingertip contacts from the user's lefthand.

Block 2604 can present a toolbar of commands while the receivingcontinues.

Block 2606 can receive a second tactile contact from the user over anindividual command of the toolbar.

Block 2608 can provide a command functionality that corresponds to theindividual command selected via the second tactile contact.

Block 2610 can allow the user to control the corresponding commandfunctionality by sliding the first tactile contact and the secondtactile contact on the touch screen.

The above described methods detail some exemplary touch screen controltechniques. The order in which the above mentioned methods are describedis not intended to be construed as a limitation, and any number of thedescribed blocks can be combined in any order to implement the methods,or an alternate method. Furthermore, the methods can be implemented inany suitable hardware, software, firmware, or combination thereof suchthat a computing device can implement the methods. In one case, themethods are stored on a computer-readable storage media as a set ofinstructions such that execution by a processor of the computing devicecauses the computing device to perform the method(s).

The techniques described in the above discussion relate to user controlvia contacting a touchscreen. These techniques are ideally suited fortouchscreen scenarios and can allow user control beyond what iscurrently available. However, at least some of these techniques offersuch an improvement over existing user control techniques that thesetechniques can be applied to other scenarios that do not includedirectly contacting a touchscreen or include direct touching incombination with indirect control techniques. For example, in somedirect touch/indirect scenarios, one hand of the user can interact witha touch pad, mouse, stylus, or some other transducer, while the user'sother hand interacts with the touch screen. In such a case, visualfeedback can be provided on the touchscreen for the location of bothhands (e.g., contact points from both hands). The user could then expandthe toolbar with the accordion motion described above, and then use asignal such as finger pressure to select from the toolbar.

An example of an indirect control scenario can include open-air gesturesrecorded with a sensor, such as a motion sensor, optical sensor, or dataglove. One commercial product that could detect such open air gesturesis Kinect® offered by Microsoft Corp. These open air gestures caninclude the accordion toolbar expansion described above. Other indirectcontrol options can include the use of touchpads, trackballs, and/orjoysticks, among others.

CONCLUSION

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

1-20. (canceled)
 21. A device comprising: a touch screen; hardwarestorage storing computer-readable instructions; and a hardwareprocessing unit configured to execute the computer-readableinstructions; wherein the computer-readable instructions, when executedby the hardware processing unit, cause the hardware processing unit to:receive a tactile contact on a region of the touch screen; display afirst command functionality on the touch screen proximate to the regionfor a predefined time; and while the tactile contact has been removedfrom the touch screen: determine that the first command functionalitydisplayed on the touch screen has not been engaged within the predefinedtime; and, responsive to a determination that the first commandfunctionality displayed on the touch screen has not been engaged withinthe predefined time, remove the first command functionality from thetouch screen and display a second command functionality on the touchscreen.
 22. The device of claim 21, wherein the tactile contactcomprises a single tactile contact entered by a finger or a stylus. 23.The device of claim 21, wherein the tactile contact includes two or moresubstantially simultaneous tactile contacts that are received within apredefined distance of one another.
 24. The device of claim 21, whereinthe first command functionality comprises multiple commands.
 25. Thedevice of claim 24, wherein the computer-readable instructions, whenexecuted by the hardware processing unit, cause the hardware processingunit to: remove all of the multiple commands of the first commandfunctionality from the touch screen responsive to determining that thefirst command functionality displayed on the touch screen has not beenengaged within the predefined time.
 26. The device of claim 25, whereinthe computer-readable instructions, when executed by the hardwareprocessing unit, cause the hardware processing unit to: gradually fadeout all of the multiple commands of the first command functionality. 27.A device comprising: a touch screen; hardware storage storingcomputer-readable instructions; and a hardware processing unitconfigured to execute the computer-readable instructions; wherein thecomputer-readable instructions, when executed by the hardware processingunit, cause the hardware processing unit to: receive a first tactilecontact and a second tactile contact in a region of the touch screen;display a first command set on the touch screen proximate to the regionwhere the first tactile contact and the second tactile contact arereceived; and when the second tactile contact moves away from the firsttactile contact, determine whether to perform a zoom action or modifythe first command set based at least on whether the first tactilecontact also moves away from the second tactile contact.
 28. The deviceof claim 27, wherein the computer-readable instructions, when executedby the hardware processing unit, configure the hardware processing unitto: in a first instance when the first tactile contact also moves awayfrom the second tactile contact, perform the zoom action; and in asecond instance when the first tactile contact is stationary, modify thefirst command set by extending the first command set on the touchscreen.
 29. The device of claim 27, wherein the first command set is atoolbar.
 30. The device of claim 29, wherein the computer-readableinstructions, when executed by the hardware processing unit, configurethe hardware processing unit to: extend or add to the toolbar byexpanding the toolbar according to movement of the second tactilecontact.
 31. The device of claim 30, wherein the toolbar is expanded inboth a horizontal and a vertical direction.
 32. The device of claim 30,wherein the toolbar is expanded in a radial fashion, an accordionfashion, or a horizontal fashion.
 33. A method performed on a devicehaving a touch screen, the method comprising: receiving a first tactilecontact and a second tactile contact on the touch screen; determining aninitial distance between the first tactile contact and the secondtactile contact; displaying a first command set on the touch screen inresponse to receiving the first tactile contact and the second tactilecontact; and detecting movement of the second tactile contact away fromthe first tactile contact; selecting an action to perform on the touchscreen in response to the movement, the action being selected based atleast on the initial distance between the first tactile contact and thesecond tactile contact; and performing the selected action.
 34. Themethod of claim 33, further comprising: selecting a zoom action ininstances when the initial distance exceeds a predetermined value; andselecting a command set modifying action in other instances when theinitial distance does not exceed the predetermined value.
 35. The methodof claim 35, further comprising: in the instances when the initialdistance exceeds the predetermined value, performing the zoom action byzooming in on content shown on the touch screen between the firsttactile contact and the second tactile contact.
 36. The method of claim35, further comprising: in the other instances when the initial distancedoes not exceed the predetermined value, expanding the first command setby adding additional commands to the first command set.
 37. The methodof claim 35, further comprising: in the other instances when the initialdistance does not exceed the predetermined value, displaying a secondcommand set concurrently with the first command set.
 38. The method ofclaim 37, further comprising: organizing multiple commands into a firstfunctional group and a second functional group; and displaying the firstfunctional group in the first command set and the second functionalgroup in the second command set.
 39. The method of claim 37, furthercomprising: organizing multiple commands into a first group ofhighly-used commands and a second group of other commands; anddisplaying the first group of highly-used commands in the first commandset and the second group of other commands in the second command set.40. The method of claim 33, wherein the movement involves movement ofboth the first tactile contact and the second tactile contact.